In many circumstances, a robot must acquire information about objects and its location within the surroundings. After the robot is placed in a new environment or the environment changes, the robot must detect objects and locate the objects using one or more sensors. Based on sensor inputs, the robot determines spatial relationship between the objects in the environment, and performs various tasks. In some robots, the robots generate occupancy maps that store coordinates of various objects in the environment. The robots then use information in the occupancy maps in conjunction with the sensor inputs to perform various operations on objects in the environment.
Many robotic manipulations such as grabbing or pushing involve actions on objects that are placed on a flat surface. Detecting and localizing objects having flat surfaces are important in many robotic applications because objects having flat surfaces (e.g., table) often serve as a supporting structure or foundation for other target objects for robotic manipulations. Therefore, the robot needs to recognize and determine the spatial relationship between the objects having the flat surface and the target object placed on the flat surface to perform certain tasks on the target object. For example, in order to grab a target object (e.g., cup) on an object having a flat surface (e.g., table), the robot must first determine the location and orientation of the object having the flats surface (e.g., table) so that the robot may move to an area where the target object (e.g., cup) can be reached.
One way of detecting an object having a flat surface is to use a camera for capturing color images of a scene. The image capturing device may generate grayscale or RGB color images. Image processing may then be performed on the captured grayscale or color images to determine the location and orientation of objects having flat surfaces. A disadvantage of using grayscale images or color images is that the accuracy of the determined location and orientation drops significantly in non-ideal cases where part of the flat surface is occluded, the edges of the flat surface are occluded, objects are cluttered on the flat surface or other objects are present in the image having features similar to the object having the flat surface.
Another way of detecting an object having a flat surface is using depth images indicating distance from a sensor to the object. A time-of-flight depth sensor is often used for generating the depth images. The time-of-flight depth sensor sends out a signal to the object and measures a property of the returned signal from the object to determine the distance to the object. The depth images obtained by measuring such returned signal are often very noisy and inaccurate. Consequently, the three-dimensional coordinates of objects determined from the depth are often unreliable and inaccurate.